A biblical plague insect could help create the future of self-driving cars

Swarms of locusts are not often associated with good omens, but when it comes to designing better vision for sophisticated computers like those inside self-driving cars, scientists say there's an important lesson to be learned.

These insects can move together in massive swarms of millions of individuals without so much as a fender bender, thanks to a specialized neuron in their brains called the Lobula Giant Movement Detector (LGMD). By designing a small, energy-efficient device that mimics this skill, a team of mechanical engineers from Penn State hopes to build a vision system that effortlessly keeps self-driving cars from colliding on the road.

Such a system could prove a crucial next step towards improving the safety of these vehicles, leading us toward a future of highways full of snoozing commuters in autonomous vehicles.

Insects are a ready source of inspiration when it comes to designing simple, effective, flight or vision systems for robotics and autonomous vehicles. Researchers have even gone as far as to design personal VR environments for flies to study their behavior and design better drones.

"Locusts are just incredible. What these creatures can do is very humbling."

Saptarshi Das, assistant professor of engineering science and mechanics at Penn State and co-author of the study, published Monday in the journal Nature Electronics, argues that locusts are unique even among insects when it comes to their vision abilities.

"We are always looking for animals with unusual abilities, ones that do something better than humans," said Das. "Insect vision is something people use regularly to design automatic systems... [s]o we started looking at how it works and, locusts are just incredible. What these creatures can do is very humbling."

While the researchers say that previous work attempting to mimic locusts' remarkable brain cells for computer systems is encouraging, the high-energy requirements and size of these systems make them impractical for effective scaling or deployment in vehicles. The team argues that their compact, energy-efficient design could break through this innovation block.

Why are these neurons so special? Part of what makes these locusts' neurons so singular, write the authors, is that they have two different ways to detect and then react to potential collisions. Locusts have a wide field of vision (in part thanks to their slightly creepy, compound eyes), allowing one branch of the LGMD neuron to "see" approaching locusts, while, another branch of the neuron "sees" the angular velocity of the incoming object, helping the locust judge how quickly it is approaching. The combined information is then pushed to another part of the locusts' brain, triggering an escape response, said Darsith Jayachandran, a graduate student in engineering science and mechanics and first author on the study.

"Because the neuron has two branches, the locust computes the changes in these two inputs and realizes that something is going to collide," said Jayachandran. "So the avoiding locust changes direction."

To mimic this in cars, the team designed a photoreceptive device just under .001 millimeters by .005 millimeters on top of a floating memory gate (a small flash memory cell). The device works by increasing its current in response to incoming light, which it interprets as an approaching object, and counterbalancing that stimuli with a decrease in current from an internal inhibitory signal.

These fight-or-flight signals are combined during a collision event to generate a non-monotonic response (a kind of artificial reasoning), imitating how a locust might suddenly decide to change course to avoid a crash.

To test how well their system works in practice, the team used a simulated car instead of the real deal. The car was able to detect an impending collision, but the limited depth and angle perception meant that the car wasn't able to anticipate which direction the threat was coming from — and, as a result, it couldn't decide how to move to avoid it.

Unlike like the locusts, which the authors say use the special neuron primarily for avoiding collisions with other locusts, the device for cars could, in theory, be used to avoid collisions with not only other vehicles but also pedestrians as well. This is of particular concern for self-driving vehicles, which, unfortunately, have a record of striking and killing pedestrians.

The researchers plan to expand the stimuli environment for their cars to include objects of different speeds, light intensities, and trajectories. They hope that these further experiments will help refine their device and improve its utility in designing avoidance collision systems for robots and autonomous vehicles.

Abstract: Accurately detecting a potential collision and triggering a timely escape response is critical in the field of robotics and autonomous vehicle safety. The lobula giant movement detector (LGMD) neuron in locusts can detect an approaching object and prevent collisions within a swarm of millions of locusts. This single neuronal cell performs nonlinear mathematical operations on visual stimuli to elicit an escape response with minimal energy expenditure. Collision avoidance models based on image processing algorithms have been implemented using analogue very-large-scale-integration designs, but none is as efficient as this neuron in terms of energy consumption or size. Here we report a nanoscale collision detector that mimics the escape response of the LGMD neuron. The detector comprises a monolayer molybdenum disulfide photodetector stacked on top of a non-volatile and programmable floating-gate memory architecture. It consumes a small amount of energy (in the range of nanojoules) and has a small device footprint (~1 μm × 5 μm).